Posterior Distribution Of Latent Variables Research Articles

Patch-Level Feature Selection for Thoracic Disease Classification by Chest X-ray Images Using Information Bottleneck.

Chest X-ray (CXR) examination serves as a widely employed clinical test in medical diagnostics. Many studied have tried to apply artificial intelligence (AI) programs to analyze CXR images. Despite numerous positive outcomes, assessing the applicability of AI models for comprehensive diagnostic support remains a formidable challenge. We observed that, even when AI models exhibit high accuracy on one dataset, their performance may deteriorate when tested on another. To address this issue, we propose incorporating a variational information bottleneck (VIB) at the patch level to enhance the generalizability of diagnostic support models. The VIB introduces a probabilistic model aimed at approximating the posterior distribution of latent variables given input data, thereby enhancing the model's generalization capabilities on unseen data. Unlike the conventional VIB approaches that flatten features and use a re-parameterization trick to sample a new latent feature, our method applies the trick to 2D feature maps. This design allows only important pixels to respond, and the model will select important patches in an image. Moreover, the proposed patch-level VIB seamlessly integrates with various convolutional neural networks, offering a versatile solution to improve performance. Experimental results illustrate enhanced accuracy in standard experiment settings. In addition, the method shows robust improvement when training and testing on different datasets.

Replica Exchange Particle-Gibbs Method with Ancestor Sampling

Particle-Gibbs (PG) method is a Markov chain Monte Carlo (MCMC) method for sampling from the joint posterior distribution of latent variables and parameters in a nonlinear state space model, which ...

A novel quadruple generative adversarial network for semi-supervised categorization of low-resolution images

In order to make utilization of unlabeled low-resolution (LR) images to shape discriminative models, we present quadruple generative adversarial network (Q-GAN), a game-theoretical framework for implementing semi-supervised categorization of LR images. It can realize photo-realistic image super-resolution (SR) and semi-supervised pattern recognition simultaneously. We consider our pipeline as a four-player optimization-based formulation, which consists of four vital components, i.e., a refiner for image SR and generation, a discriminator for identifying high-resolution (HR) samples and another for identifying true (original) samples, a classifier for label prediction. The refiner and two discriminators characterize the conditional distributions between images and labels, whilst the classifier solely focuses on predicting real image-label pairs. We select those high-quality super-solved images with ground-truth labels for data supplement. We customize the global optimization objective function as well as the training procedure to ensure model approximates the posterior distribution of latent variables given true data in a semi-supervised manner. Experimental results demonstrate that Q-GAN can simultaneously (1) deliver the promising categorization performance among state-of-the-arts, i.e., validation accuracy achieves 92.18% and testing accuracy achieves 90.63%, and (2) recover fine-grained textures with high peak signal-to-noise ratios (PNSRs) and structural similarities (SSIMs) from heavily downsampled testing images of hand-crafted dataset and public benchmarks.

Multiview learning with variational mixtures of Gaussian processes

Gaussian processes (GPs) are powerful Bayesian nonparametric tools widely used in probabilistic modeling, and the mixture of GPs (MGPs) were introduced afterwards to make data modeling more flexible. However, MGPs are not directly applicable to multiview learning. In order to improve the modeling ability of MGPs, in this paper, we propose a new framework of multiview learning for the MGPs and instantiate it for classification. We make the divergence between views as small as possible while ensuring that the posterior probability of each view is as large as possible. Specifically, we regularize the posterior distribution of latent variables with the consistency of posterior distributions of the latent functions between different views. Since it is intractable to solve the model analytically, the variational inference and optimization algorithms of the classification model are also presented in this paper. Experimental results on multiple real-world datasets have shown that the proposed method has outperformed the original MGP model and several state-of-the-art multiview learning methods, which indicate the effectiveness of the proposed multiview learning framework for MGPs.

A probabilistic framework to incorporate mixed-data type features: Matrix factorization with multimodal side information

Recommender systems that exclusively rely on past interactions between the users and the items underperform in settings with very few observations. Their performance at such extreme sparsity can be improved by exploiting the side information of the users and the items including the demographics and the item descriptions. However, such side information is mostly heterogeneous and multimodal, including both numerical and categorical features to yield a non-trivial incorporation process. Researchers have addressed this problem mainly by converting the categorical features into numerical ones or forming numerical similarity matrices. This paper presents a different approach in the form of a novel Bayesian probabilistic generative framework which can effectively incorporate multimodal side information into the matrix factorization (MF-MSI) model. With the help of local quadratic bounds on the categorical likelihoods, we derive a scalable and computationally efficient iterative optimization method based on the variational EM to learn the posterior distributions of latent variables for both the users and the items. A comprehensive experimental study on both simulated and real benchmark datasets demonstrate the proof-of-concept where the additional side information improves both the prediction accuracy and the ranking performance over more than a dozen popular baseline models. Finally, the proposed MF-MSI model claims state-of-the-art performance in the majority of the test scenarios when compared to more recently introduced recommender systems which can also utilize the side information via different techniques.

Fast Bayesian inference using Laplace approximations in a flexible promotion time cure model based on P-splines

Bayesian methods for flexible time-to-event models usually rely on the theory of Markov chain Monte Carlo (MCMC) to sample from posterior distributions and perform statistical inference. These techniques are often plagued by several potential issues such as high posterior correlation between parameters, slow chain convergence and foremost a strong computational cost. A novel methodology is proposed to overcome the inconvenient facets intrinsic to MCMC sampling with the major advantage that posterior distributions of latent variables can rapidly be approximated with a high level of accuracy. This can be achieved by exploiting the synergy between Laplace’s method for posterior approximations and P-splines, a flexible tool for nonparametric modeling. The methodology is developed in the class of cure survival models, a useful extension of standard time-to-event models where it is assumed that an unknown proportion of unidentified (cured) units will never experience the monitored event. An attractive feature of this new approach is that point estimators and credible intervals can be straightforwardly constructed even for complex functionals of latent model variables. The properties of the proposed methodology are evaluated using simulations and illustrated on two real datasets. The fast computational speed and accurate results suggest that the combination of P-splines and Laplace approximations can be considered as a serious competitor of MCMC to make inference in semi-parametric models, as illustrated on survival models with a cure fraction.

A Probabilistic Just-in-Time Learning Framework for Soft Sensor Development With Missing Data

Just-in-time learning (JITL) is one of the most widely used strategies for soft sensor modeling in nonlinear processes. However, traditional JITL methods have difficulty in dealing with data samples that contain missing values. Meanwhile, data noises and uncertainties have not been taken into consideration for relevant sample selection in existing JITL approaches. To overcome these problems, a new probabilistic JITL (P-JITL) framework is proposed in this brief. In P-JITL, variational Bayesian principal component analysis is first utilized to handle missing values and extract Gaussian posterior distributions of latent variables. Then, symmetric Kullback-Leibler divergence is creatively employed to measure the dissimilarity of two distributions for relevant sample selection in the JITL framework. Finally, a nonlinear regression model, Gaussian process regression, is carried out to model the nonlinear relationship between the output and the extracted latent variables. In this way, the proposed probabilistic JITL (P-JITL) is able to deal with missing data and select relevant samples more accurately. To evaluate the effectiveness and flexibility of P-JITL, comparative studies between P-JITL and traditional deterministic JITL (D-JITL) are carried out on a numerical example and an industrial application example, in which missing data are simulated with percentages from 0% to 50%. The results show that P-JITL can provide more accurate prediction accuracy than D-JITL in each scenario considered.

Bayesian spatial prediction of skew and censored data via a hybrid algorithm

A correct detection of areas with excess of pollution relies first on accurate predictions of pollutant concentrations, a task that is usually complicated by skewed histograms and the presence of censored data. The unified skew-Gaussian (SUG) random field proposed by Zareifard and Jafari Khaledi [19] offers a more flexible class of sampling spatial models to account for skewness. In this paper, we adopt a Bayesian framework to perform prediction for the SUG model in the presence of censored data. Owing to the presence of many latent variables with strongly dependent components in the model, we encounter convergence issues when using Monte Carlo Markov Chain algorithms. To overcome this obstacle, we use a computationally efficient inverse Bayes formulas sampling procedure to obtain approximately independent samples from the posterior distribution of latent variables. Then they are applied to update parameters in a Gibbs sampler scheme. This hybrid algorithm provides effective samples, resulting in some computational advantages and precise predictions. The proposed approach is illustrated with a simulation study and applied to a spatial data set which contains right censored data.

Estimation in Truncated GLG Model for Ordered Categorical Spatial Data Using the SAEM Algorithm

In this article, we utilize a scale mixture of Gaussian random field as a tool for modeling spatial ordered categorical data with non-Gaussian latent variables. In fact, we assume a categorical random field is created by truncating a Gaussian Log-Gaussian latent variable model to accommodate heavy tails. Since the traditional likelihood approach for the considered model involves high-dimensional integrations which are computationally intensive, the maximum likelihood estimates are obtained using a stochastic approximation expectation–maximization algorithm. For this purpose, Markov chain Monte Carlo methods are employed to draw from the posterior distribution of latent variables. A numerical example illustrates the methodology.